meson2hermetic: the world's premiere build system converter by gurchetansingh · Pull Request #15462 · mesonbuild/meson

gurchetansingh · 2026-01-12T22:09:24Z

This API converts meson.build into their corresponding hermetic representation (Android Blueprint, Bazel and maybe Buck2 in the future).

This is motivated by the need to integrate Mesa3D into AOSP and Fuchsia trees.

The question "how to build and update Mesa3D drivers for Android?" [1] in particular has led to several methods over the years, none of which used Android's native build system (Soong). This has been an obstacle to adoption of open-source drivers, which everyone knows are more secure, maintainable and faster than closed alternatives.

By integrating into Mesa3D's native build system (Meson), "meson convert" brings shocking and jaw-dropping clarity to the question.

Technically speaking, the tool works via a series of TOML files. Python 3.11 has tomlib in the standard library, and mconvert.py uses a conditional import strategy to prevent issues on older Python versions.

These TOML files specify:

the Meson project that is being converted
where to find the dependencies in a hermetic tree
which compilers that a hermetic tree supports

These TOML files are used to run the Meson interpreter multiple times. For example, the set of C/C++ flags may be different if the compiler targets x86_64 or ARM64. Data from each of of the meson interpreter runs is collated to reconstruct the full set of Soong/Bazel rules.

Although the initial implementation is focused with converting to a existing hermetic build system, this introduces infrastructure that could be useful if Meson itself takes a look at remote-executed, hermetic builds. For example, Meson can download prebuilts from NixPkgs.

[1] https://gitlab.freedesktop.org/mesa/mesa/-/issues/13776

eli-schwartz · 2026-01-12T22:21:25Z

As relevant context, this tool was previously suggested by @gurchetansingh at #14134

/cc @dcbaker based on past discussions.

eli-schwartz · 2026-01-12T22:36:20Z

I haven't yet had an opportunity to take a close look at this. I see that it adds two new commands, one being the command to actually generate the hermetic buildsystem e.g. bazel, and another "check-toolchain" command that appears to be similar in spirit to the existing env2mfile command.

I'd like to hear a high-level explanation for what this tool does, to help better understand it. (And in particular, what are the challenges that make using machine files directly, not sufficient for your purposes? It looks like maybe the main differentiator is a lack of support for specifying the results of cc.has_* checks, but if that's the case then I'd like to implement that for machine files anyway...)

gurchetansingh · 2026-01-12T23:21:06Z

For reference, you can run the tool given the directions at:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39279

Right now, only mesa3d is supported.

I'd like to hear a high-level explanation for what this tool does, to help better understand it. (And in particular, what are the challenges that make using machine files directly, not sufficient for your purposes? It looks like maybe the main differentiator is a lack of support for specifying the results of cc.has_* checks, but if that's the case then I'd like to implement that for machine files anyway...)

One thing I like about .toml is the ability to use comments and have multiple tool-chains (I needed 6 tool-chains in the end to generate the right hermetic rules) in the same file, since IIUC is missing for the .ini files. Essentially, it seems to be the machine-files are just pared down versions of toml, which of course makes sense pre-Python 3.11. But with Python 3.11 on the horizon, using toml was very convenient and feature rich.

dcbaker · 2026-01-13T17:59:25Z

I will make a point to look at this in the near future.

As a bit of an aside: I'm stilling working on the ability to have meson definitions for LLVM, but I need to get more changes into Meson's dependency representation to be able to model that.

dcbaker · 2026-01-13T18:00:45Z

One thing I saw implementation wise from a quick glance, the new convert package has not been added to run_mypy.py, so we're not actually getting type checking. I'd really like to see that turned on for new code.

bonzini · 2026-01-14T15:40:30Z

@gurchetansingh please add stubs to docs/markdown/Commands.md so that CI passes.

To build the documentation you need to install hotdoc, then you can use

          cd docs
          ../meson.py setup _build
          ninja -C _build
          ninja -C _build test

gurchetansingh · 2026-01-15T18:29:37Z

As a bit of an aside: I'm stilling working on the ability to have meson definitions for LLVM, but I need to get more changes into Meson's dependency representation to be able to model that.

Nice. LLVM to hermetic is challenging as well like Mesa. Plus, I think many devs would enjoy an improvement over CMake.

convert package has not been added to run_mypy.py

Added: convert_interpreter.py is failing the CI though due to the current type checking config. That's understandable -- I added questionable code to pass run_mypy.py. BUT, when I remove questionable code, the script fails:

  File "meson/mesonbuild/convert/convertmain.py", line 26, in <module>
    from mesonbuild.convert.convert_interpreter import ConvertInterpreter
  File "meson/mesonbuild/convert/convert_interpreter.py", line 18, in <module>
    from mesonbuild.interpreter import kwargs
  File "meson/mesonbuild/interpreter/kwargs.py", line 15, in <module>
    from ..dependencies.base import Dependency, DependencyMethods, IncludeType
ImportError: cannot import name 'IncludeType' from 'mesonbuild.dependencies.base' (meson/mesonbuild/dependencies/bas

But should IncludeType only be required with T.TYPE_CHECKING? Could be related to recent refactors.

dcbaker · 2026-01-15T19:11:12Z

@gurchetansingh: interpreter/kwargs.py should only be imported inside a T.TYPE_CHECKING block. Or, at least it currently has that assumption.

dcbaker

I've started looking purely at the implementation details of this, but I haven't looked too much at the design yet. It's a lot of code and this is giving me a chance to try to get familiar with it.

I haven't looked at the soong implementation yet, and I'm about half way through the instance code of the third patch. I still need to get to reference/ and to top level convert/ file and mconvert.py

One thing I'd really like to have is some more comments, particularly docstrings to help explain how all of the code fits together, and what each thing is for, there's a lot of code here and it would really help me to review it (and for us to maintain it).

mesonbuild/mchecktoolchain.py

dcbaker · 2026-01-15T19:18:41Z

mesonbuild/mchecktoolchain.py

+    parser.add_argument("-o", "--output", default=None, help="Output file name.")
+
+
+def run(options: argparse.Namespace) -> int:


We do have some uses of Protocol in these front end functions to help with accurate typing vs using Namespace, I'm not sure how practical that is in this case.

left as is since most other commands use namespace

mesonbuild/check_toolchain/android.py

dcbaker · 2026-01-15T19:23:18Z

mesonbuild/convert/abstract/abstract_toolchain.py

+class AbstractCompiler(Compiler):
+    def __init__(self, conf: T.Dict[T.Any, T.Any], *args: T.Any, **kwargs: T.Any):
+        self.conf = conf
+        super().__init__(*args, **kwargs)


I'm not crazy about the heavy use of Any here, but I need to think more about whether there's another good option.

dcbaker · 2026-01-15T19:26:28Z

mesonbuild/convert/abstract/abstract_toolchain.py

+    def find_library(
+        self,
+        libname: str,
+        extra_dirs: T.List[str],
+        libtype: LibType = LibType.PREFER_SHARED,
+        lib_prefix_warning: bool = True,
+        ignore_system_dirs: bool = False,
+    ) -> T.Optional[T.List[str]]:


Please do not use the black style especially this thing it does, it's a really weird style for python and we don't use it elsewhere, the ) -> T.Optional[...]: should not be dedented tot he same level as def

one of:

def find_library(self, libname: str, ...): ... def find_library(self, libname: str, ...): ... def find_library( self, libname: str, ...): ...

Would be consistent with our style elsewhere.

I added a simple .style.yapf based on pep8 . It dedents to the same level as the function definition much less than ruff (the prior tool I used). It does in a few select cases, but I think those cases are reasonable. LMK otherwise, and I can skip the linters and hand-code to match the style you want.

mesonbuild/convert/instance/convert_instance_custom_target.py

gurchetansingh · 2026-01-16T20:56:02Z

One thing I'd really like to have is some more comments, particularly docstrings to help explain how all of the code fits together, and what each thing is for, there's a lot of code here and it would really help me to review it (and for us to maintain it).

Added more docstring comment. I would recommend reviewers start in convertmain.py.

This adds the following API: meson check-toolchain [ARGUMENTS] This API checks the the specified toolchains and outputs a TOML file that contains an abstract reprenstation of it's properties. An example TOML output is: [[toolchain]] name = "android_arm64" [toolchain.host_machine] cpu_family = "aarch64" cpu = "aarch64" system = "android" endian = "little" [toolchain.c] compiler_id = "clang" linker_id = "ld.lld" version = "21.0.0" [toolchain.c.links.fails] "GCC 64bit atomics" = true [toolchain.c.check_header.fails] "pthread_np.h" = true [toolchain.c.has_header_symbol.fails] "sys/mkdev.h" = { major = true, minor = true, makedev = true } "errno.h" = { program_invocation_name = true } [toolchain.c.has_function.fails] qsort_s = true secure_getenv = true The has_function(..) or check_header(..) logs failures, since if one logged the successes, the resultant file would be very long and hard to read. The checks that the toolchain currently performs are modeled on Mesa3D's meson.build file, but can be be a superset of checks needed by the users of the tool in the future. The first user of the tool is meson convert, which is added in a subsequent patch. There are fast paths for for Android + Fuchsia: * meson check-toolchain --android-ndk-version r29 -o android_r29.toml * meson check-toolchain --fuchsia-clang-instance-id <instance ID mesonbuild#1> \ --fuchsia-core-sdk-instance-id <instance ID mesonbuild#2> \

The convert tool will use it to generate build targets. It is possible for there to be a dedicated convert "backend" which passes in context like vslite_ctx, but that would require: - Made the core changes more invasive - Split the convert code across more directories, which may be logically harder to follow

This API converts meson.build into their corresponding hermetic representation (Android Blueprint, Bazel and maybe Buck2 in the future). This is motivated by the need to integrate Mesa3D into AOSP and Fuchsia trees. The question "how to build and update Mesa3D drivers for Android?" [1] in particular has led to several methods over the years, none of which used Android's native build system (Soong). This has been an obstacle to adoption of open-source drivers, which everyone knows are more secure, maintainable and faster than closed alternatives. By integrating into Mesa3D's native build system (Meson), "meson convert" brings shocking and jaw-dropping clarity to the question. Technically speaking, the tool works by via series of TOML files. Python 3.11 has tomlib in the standard library, and mconvert.py uses a conditional import strategy to prevent issues on older Python versions. These TOML files specify: * the Meson project that is being converted * where to find the dependencies in a hermetic tree * which compilers that a hermetic tree supports These TOML files are used to run the Meson interpreter multiple times. For example, the set of C/C++ flags may be different if the compiler targets x86_64 or ARM64. Data from each of meson intrepreter runs is collated to reconstruct the full set of Soong/Bazel rules. Although the initial implementation is focused with converting to a existing hermetic build system, this introduces infrastructure that could be useful if Meson itself takes a look at remote-executed, hermetic builds. For example, Meson can download prebuilts from NixPkgs or grow the ability to handle other projects. [1] https://gitlab.freedesktop.org/mesa/mesa/-/issues/13776 Co-developed-by: Craig Stout <cstout@google.com> Co-developed-by: Brandon Nguyen <bpnguyen@google.com>

For convert and check-toolchain: - Add to mesonmain.py - Add to run_mypy.py - Add Commands.md

To ensure that the workflow is not broken, add a few unit test. The test: 1) runs meson check-toolchain with a specified output file and name, targeting the native toolchain. 2) parses the generated TOML file. 3) verifies that the output contains common data - host_machine details (like cpu_family and system) - compiler information for both C and C++ Test command: - python3 run_unittests.py CheckToolchainTests

gurchetansingh · 2026-03-16T16:23:57Z

Changelog since last update

1. meson-to-Bazel for Fuchsia

A Bazel backend functional enough to compile gfxstream_vk for Fuchsia.

If you have:

Bazel installed (https://bazel.build/install)
and the Mesa MR (https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40443)
this MR

here is the command to test:

>> ~/mesa3d $ python3 ~/meson/meson.py convert fuchsia mesa3d
>> ~/mesa3d $ bazel build //src/gfxstream/guest/vulkan:vulkan_gfxstream --platforms=//bazel/platforms:fuchsia_x86_64_platform --host_platform=@platforms//host

2. Enhanced meson check-toolchain

Now supports downloading from a [compiler_binaries.wrap] section with standard parameters (source_url, source_hash, source_filename).

3. Unit tests for CI/CD

To make sure the tool works as expected. Looks like the unittests are failing on platforms without Python 3.11, and that will be addressed in a future update.

4. formatter issues fixed

This style uses ruff as the formatter, but it's turned off for parts Meson developers especially hate (like the function defintions). One thing that would be nice if Meson had an officially supported formatter or style file (regardless of which one it is).

meson2hermetic: a guide for reviewers

At its core, meson2hermetic is a two-step build system transpiler.

1. meson check-toolchain

This tool probes a toolchain (like the Android NDK or Fuchsia SDK) to see which headers, functions, and flags are supported, saving the results for the conversion process.

    DOWNLOADS               EXECUTION                OUTPUT
 +--------------+    +-----------------------+    +----------------+
 |  Android NDK |    |  Meson Introspection  |    |                |
 |      OR      |--->|  & Compiler Probing   |--->| toolchain.toml |
 |  Fuchsia SDK |    | (check_header, etc.)  |    |                |
 +--------------+    +-----------------------+    +----------------+
                                ^
                                |
                         +--------------+
                         |  Cross File  |
                         +--------------+

2. meson convert

This tool takes a standard Meson project and translates it into a different build system (Soong or Bazel) using the metadata gathered by the toolchain checker.

     INPUTS                   PROCESSING                    OUTPUTS
 +-------------+      +----------------------+      +----------------+
 | meson.build |      |   Custom Abstract    |      |   Android.bp   |
 +-------------+      |   Meson Interpreter  |      |    (Soong)     |
        +             +----------------------+      +----------------+
 +-------------+                 |                          OR
 | project.toml|                 v                  +----------------+
 +-------------+      +----------------------+      |   BUILD.bazel  |
        +             |   Build-Agnostic     |      |     (Bazel)    |
 +-------------+      |   State Tracker      |      +----------------+
 |toolchain.toml|     +----------------------+              ^
 +-------------+                 |                          |
                                 v                  +----------------+
                      +----------------------+      |    Emitter     |
                      | Dependency Resolving |----->|    Backend     |
                      +----------------------+      +----------------+

FAQ

1. Does the tool handle the entire Meson API?

Not yet. build.Executable is one notable area where I haven't had a way to test yet (but it's on the radar). configuration-data too

2. How stable are the new APIs?

The tool is useful for developers integrating FOSS projects into gigantic monorepos. This is a niche audience. So the tool's API will likely be considered "experimental" and subject to change for a while (six months to 1 year).

Specifically, this flow uses TOML files to handle multiple compilers/sysroots, but I'm unsure if the approach is exactly right (though I don't think Bazel handles that well either). If a better representation emerges, this tool will likely migrate.

3. Is this only useful for Mesa3D?

Some people also need hermetic QEMU builds, and someone was playing around with DPDK + Bazel (magma-gpu/mesonbuild#2). It's a niche audience, but not limited to Mesa3D alone.

4. Why not use AI to transpile?

The project was conceived before generative AI tools were widely available. But for the same reasons clang/gcc are better than AI-to-bytecode conversion, a dedicated tool offers better utility. Specifically:

AI is not deterministic. Depending on model/prompt/infra, different results.
It probably won't work without anything that constrains the search, like the TOML files do in this case. Creating the harness could be just as complex as this tool.
Where will the AI harness be run?
For a first pass, AI could take days of trial and error to achieve the initial conversion. Incremental updates with AI would be easier though.
AI token prices are currently heavily subsidized, and prices may rise in the future.

This adds CI/CD for 'meson convert'. Idea is simple: - have meson.build files - have "golden" Soong/Bazel files - run meson convert - compare with the goldens. Test command: - python3 run_unittests.py ConvertTests

gurchetansingh requested a review from jpakkane as a code owner January 12, 2026 22:09

gurchetansingh force-pushed the meson2hermetic branch from c9ebe87 to d3dff63 Compare January 12, 2026 22:13

eli-schwartz added enhancement android labels Jan 12, 2026

gurchetansingh force-pushed the meson2hermetic branch from d3dff63 to f8081d8 Compare January 15, 2026 17:49

dcbaker requested changes Jan 15, 2026

View reviewed changes

gurchetansingh force-pushed the meson2hermetic branch from f8081d8 to a2f97c3 Compare January 16, 2026 20:11

gurchetansingh force-pushed the meson2hermetic branch from a2f97c3 to ac3a82c Compare March 16, 2026 16:06

gurchetansingh and others added 5 commits March 16, 2026 09:08

meson: core changes to enable new APIs

27a6eca

For convert and check-toolchain: - Add to mesonmain.py - Add to run_mypy.py - Add Commands.md

gurchetansingh force-pushed the meson2hermetic branch from ac3a82c to 65b375a Compare March 16, 2026 16:11

meson: convert: add CI/CD + unit tests

b1c12f5

This adds CI/CD for 'meson convert'. Idea is simple: - have meson.build files - have "golden" Soong/Bazel files - run meson convert - compare with the goldens. Test command: - python3 run_unittests.py ConvertTests

gurchetansingh force-pushed the meson2hermetic branch from 65b375a to b1c12f5 Compare March 16, 2026 17:54

		parser.add_argument("-o", "--output", default=None, help="Output file name.")


		def run(options: argparse.Namespace) -> int:

Uh oh!

Conversation

gurchetansingh commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eli-schwartz commented Jan 12, 2026

Uh oh!

eli-schwartz commented Jan 12, 2026

Uh oh!

gurchetansingh commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dcbaker commented Jan 13, 2026

Uh oh!

dcbaker commented Jan 13, 2026

Uh oh!

bonzini commented Jan 14, 2026

Uh oh!

gurchetansingh commented Jan 15, 2026

Uh oh!

dcbaker commented Jan 15, 2026

Uh oh!

dcbaker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dcbaker Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

gurchetansingh Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dcbaker Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

dcbaker Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

gurchetansingh Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gurchetansingh commented Jan 16, 2026

Uh oh!

gurchetansingh commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog since last update

1. meson-to-Bazel for Fuchsia

2. Enhanced meson check-toolchain

3. Unit tests for CI/CD

4. formatter issues fixed

meson2hermetic: a guide for reviewers

1. meson check-toolchain

2. meson convert

FAQ

1. Does the tool handle the entire Meson API?

2. How stable are the new APIs?

3. Is this only useful for Mesa3D?

4. Why not use AI to transpile?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gurchetansingh commented Jan 12, 2026 •

edited

Loading

gurchetansingh commented Jan 12, 2026 •

edited

Loading

gurchetansingh commented Mar 16, 2026 •

edited

Loading